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ABSTRACT 

Intended to assist underachieving high school pupils 
in improving their reading and communication skills, the Secondary 
Developmental Reading Program in the Columbus, Ohio Public Schools, 
emphasized literacy survival skills and reached 809 pupils in grades 
9 and 10 in 13 senior high schools during the 1986-87 school year. 
The program used diagnostic testing to assess pupils' individual 
reading strengths and weaknesses, and provided individualized 
instruction to meet pupils' needs on a daily basis in a small group 
setting. Some program sites used computer-assisted instruction as 
well. Success of the program was judged by results of the 
"Comprehensive Test of Basic Skills" Reading Comprehension Subtest as 
pretest and posttest, and the General Inservice Evaluation Form, 
developed locally. Test results showed that none of the program's 
objectives were met — pupils with good attendance records did not 
improve their reading scores, nor did the target of 80% of program 
personnel evaluate the program as successful. The computer users 
scored no better than other groups on the achievement criterion. 
Recommendations included rescheduling the program, giving more 
attention to students' individual needs, and showing greater 
commitment to the program on the part of teachers and administrators 
at participating schools. (References and copies of the evaluation 
materials are appended.) (SKC) 
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FINAL EVALUATION REPORT 
LANGUAGE DEVELOPMENT COMPONENT 
SECONDARY DEVELOPMENTAL READING 
1966-87 

ABSTRACT 



Program Description: The Secondary Developmental fading (SDR) Program served 
809 pupils in grades 9-10 in 13 senior high schools. Funding of the component 
was made available through the Ohio Disadvantaged Pupil Program Fund (CPPF). 

The purpose of the SDR Program is to assist underachieving high school 
pupils in raising their reading and communication skills. Emphasis of the 
program is placed on literacy survival skills necessary to function in our 
T ord-oriented world. 

Within the 1986-87 SDR Program eight teachers in eight senior high schools 
participated in a project which utilized Apple computers for computer assisted 
instruction/computer management system (CAI/CMS). The computers, software, and 
attendant services were contracted with the Prescription Learning (PL) Company 
of Springfield, Illinois. The regular treatment group had six teachers in five 
senior high schools. 

Time Interval: For evaluation purposes, the Secondary Developmental Reading 
Program started on September 15, 1986 and continued through April 3, 1987. 
This interval of time gave 129 possible days of program instruction. Pupils 
included in the final pretest-posttest analysis must have attended at least 103 
days (80%) during the tiT*e period stated above. 

Activities: The program made use of diagnostic testing to assess pupils" 
individual reading strengths and weaknesses. Individualized instruction to 
meet pupils' needs was provided on a daily basis in a small group setting. 

Program Objectives: The program had two objectives. Objective 1.1 stated that 
an evaluation sample will be comprised of pupils who score at or below the 
36%ile on a selection test and are in attendance at least 80% of the 
instructional period. Pupils who attend 80% of the 6.5 month treatment period 
will show an average gain in reading of 1.0 NCE for each month, which is an 
average gain of 6.5 NCE's overall (6.5 months x 1.0 NCE). Objective 2.1 stated 
that program personnel will be provided at least two inservice sessions and 
that at least 80% of the personnel attending each sessioft will rate the session 
as valuable in providing information that will assist them in carrying out 
their program responsibilities. 

Evaluation Design: Objective 1.1 was evaluated through the administration of 
the Comprehensive Tests of Basic Skills (CTBS) Reading Comprehension subtest. 
Analyses of the data included comparison of pretest to posttest change scores 
in terms of grade equivalents, percentiles, and NCR's. Objective 2.1 was 
evaluated by means of the General Insorvice Evaluation Form, a locally 
constructed instrument. 
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Major Findings/Recommendations; The information collected on the Pupil Census 
Forms indicated the program served 809 pupils for an average of 3.5 hours of 
instruction per week. The average daily membership in the program was 660.5 
pupils. Tne average days* of earollmeat per pupil was 105.3 days and the 
average attendance per pupii was 86.8 days. The average number of pupils 
served per teacher was 47.2. 

Objective 1.1, that pupils who attended 80% of the 6.5 month treatment 
period would show an average gain in reading of 1.0 NCE for each month, was not 
attained. There was a negative average change of -8.3 or -1.3 NCE/monr.h. 

Objective 2.1, that program personnel would be provided ac least two 
inservice sessions and that at least 80% of the personnel attending each 
session would rate the session as valuable in providing information to assist 
them in carrying out their program responsibilities , was not attained because 
one of the two sessions was rated as valuable in carrying out component 
responsibilities by less than the requisite 80% of the participants. 
Furthermore, only one of the two sessions was available to all SDR teachers, 
the other session being specifically for CAI/CMS teachers. When a combined 
rating of both inservice sessions was computed, an overall average of 88.9% of 
the participants rated the inservice sessions as valuable in carrying out 
component responsioilities . 

The CAI/CMS project was located in eight high schools. The computer 
assisted units served 486 pupils. Neither the CAI/CMS project group nor the 
group receiving regular program instruction attained the achievement 
criterion. The CAI/CMS group had a negative change of -9.2 NCE's in a 6.5 
month period, while the regular group had a negative change of -6.7 NCE's. 

A cost-benefit study indicated that cost per pupil was greater and NCE 
gains smaller in the CAI/CMS group i_han in the regular group. Based on average 
daily membership, the cost per pupil was $1180.76 in the CAI/CMS group and 
$894.37 in the regular group. Differences in NCE gains were noted above. 
However, CAI/CMS teachers served an average of 8.4 more pupils per teacher than 
in the regular group, based on average daily membership, and attendance v:as 
somewhat better in the CAI/CMS group. 

The following program recommendations were made: (a) make the program an 
elective course for pupils with selection test scores below the 36th 
percentile; (b) provide more inservice, geared specif icaJly to the high school 
level, with special stress on reading comprehension; (c) schedule time for 
cooperative planning between program and classroom teachers, to direct program 
instruction toward content area of pupil's greatest need; (d) study ways to 
increase parent involvement; (e) identify problems in the CAI/CMS part of the 
program and try to find solutions; (f) review selection procedures, correlation 
of course content to system's Course of Study , instructional methods , class 
size, and test content to determine why pupils are not 3howing desired growth; 
(g) school administrators and staff should take the responsibility of assuring 
an optimum testing environment by not scheduling unsuitable activities during 
testing weeks and adjusting class schedules to accommodate the length of the 
tests; (h) keep conditions for the pretest and for the posttest as comparable 
as possible; (i) conduct a study by giving ninth grade pupils the standard 
version of the posttest as well as the customized version, to determine 
comparability of resultant test scores; and (j) conduct an extensive review of 
the program to determine whether it should be continued in present form, 
modified, or discontinued. 



EVALSRVCS/P51 0/SDRABST87 



4 



Onio Disadvantaged Pup'' . Program Fund 



FINAL EVALUATION REPORT 
LANGUAGE DEVELOPMENT COMPONENT 
SECONDARY DEVELOPMENTAL READING PROGRAM 

July 1987 



Pro gram Descri ptio n 

The Secondary Developmental Reading (SDR) program began in the Columbus 
Public Schools in the fall oi 1971 as a component of the Ohio Disadvantaged 
Pupil Program Fund. The 1986-87 version of the SDR program was located in 13 
Columbus senior high school buildings. Fourteen project reading teachers 
worked in these 13 schools with 809 pupils in grades 9-10 who scored at or 
below the 36th percentile on a standardized achievement test in reading used 
tor selection purposes. 

Within tue l9tf6-8/ SDR program eight, teachers in eight senior high schools 
participated in a project which utilized Apple computers for computer assisted 
instruction/ computer management system (CAI/CMS). The computer software, and 
attendant services were contracted with the Prescription Learning (PL) Company 
or Springfield, Illinois. In addition to providing a new technique to reading 
and language iustruction, the use of CAI/CMS was intended to enable teachers to 
serve more pupils than would be possible in regular SDR classrooms. The use of 
CAI/CMS was also intended to be a cost-effective alternative to replacing badly 
worn conventional equipment. Of the 809 pupils in the SDR program, 486 
received computer assisted instruction and 323 received regular SDR program 
instruction. 

The purpose of the SDR program was to assist underachieving senior high 
pupils in raising their reading and communication skills. Emphasis of the 
program was placed on literacy survival skills necessary to function in our 
word-oriented world. 



Features of the SDR program included the following: 



1. 


Diagnostic testing to assess a pupil's 
reading strengths and weaknesses. 


individual 




2. 


Individualized instruction tailored to 


meet the needs 


of pupils. 


3. 


Small group instruction. 






4. 


On-going evaluation of pupils to assess 


their reading 


needs • 


5. 


Inservice meetings for teachers. 
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Ob j e ct i ve j . 1 An evaluation sample will be comprised of pupils who score at or 
below the 36£ile on a selection test and are in actendance at least 80% of the 
instructional period. The* average reading growth of pupils in the evaluation 
sample and participants in the Computer Abbisteu Instruct ion/Computer 
Management System (CAI/CMS) will be 1 .0 normal curve equivalent (NCE) point for 
each month of instruction. 

The program time period established for evaluation purposes was 129 days 
beginning September IT, 1986, and ending April 3, 1987. This time period (129 
days divided by an average of 20 scnool days per month) is equal to 6.5 
possible months of instruction. Analysis of pretest-postteot performance was 
contingent on pupil attendance for 103 days {80%) of the 129 day period. 

Objective 2 .1 To provide at least two inservice sessions to program personnel 
such that at least 80% of the inservice participants will rate each session as 
valuable in providing information that will assist them in carrying out their 
program responsibilities. 



Evaluation Design 

Trie evaluation design for tae SDR program called for the collection of data 
in three areas. 

1. Pupil Census Information 

The Pupil Census Form was developed for the purpose of 
collecting pupil demographic and participation data in the 
Secondary Developmental Reading Program (SDR). Project 
tachers maintained th*> Pupil Census Forms for all pupils 
throughout the school year or when the pupils left the 
program. Data collected on the Pupil Census Forms were the 
number of days the pupil was enrolled in the program, the 
number of days the pupil was in attendance, and the average 
number of hours per week the project teacher served the 
pupil. Other information collected included the pupil's grade 
and sex, identification of non-English speaking pupils, 
identification of any pupil wt;o left the DPPF program because 
of qualifying for a special education program, and a question 
regarding a pupil's progress which required a subjective 
response from the project teacher. A copy of the Pupil Census 
Form can be found in the Appendix. 

2. Standardized Achievement Test Information 

The purpose of the administration of the standardized 
achievement test was to collect pretest-posttest achievement 
data on all SDR program pupils to determine if Objective 1.1 
was achieved. The standard achievement test used was the 
Comprehensive Tests of Basic Skills (CTBS) , Rea ing 
Comprehension (CTB-McGraw Hill, 1981). The CTBS Reading 
Comprehension tests wer*a administered on September 22 
September 26, 1986, and again on April 6-10, 1987. The 
following lists the form, subtest and test levels of the CTBS 
used for each grade level. 
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Grade Subtest Prete st Posttest 

9 Reading Comprehension Level J Form U Level J Form V* 

10 Reading Comprehension Level J Form U Level J For;r V 

*Estimated by administration of customized Form V. 

At posttest time , grade nine was administered a customized test which 
included items yielding criterion-referenced scores in addition to a 
customized form of the norm-referenced test. The customized tests were 
developed by Columbus Public Schools personnel in cooperation with 
CTB/McGraw Hill to match the Columbus Public Schools Graded Course of 
Study. 

The achievement tests were administered as follows: Pretests for grades 
9-10 were administered by program teachers. Posttests for grade 9 were 
administered as part ol Districtwide Testing. Grade 10 was one of the 
exceptions to Districtwide Testing, aad teachers of grade 10 pupils had 
to administer their own posttests. During Districtwide Testing, tests 
were administered by classroom teachers with program teachers serving as 
proctors in some classrooms. Pretesting occurred during the week of 
September 22 - September 26, 1986; posttesting occurred April 6-10, 
1987. 

3. Inservice Evaluation 

The locally developed General Inservice Evaluation Form was designed to 
obtain teacher perceptions regarding e*ch inservice sessior The form 
was administered to participants at the close of inservice sessions. A 
modified version of the form was used for the orientation meeting of 
September 2, 1986, which was attended by regular SDR and CAI/CMS 
teachers. There was a total of two inservice meetings - one of which 
was available to regular SDR teachers and two of which were available to 
SDR teachers in the CAI/CMS project. The dates and topics of inservice 
sessions in the 1986-87 school year were as follows: 

September 2, 1986 Opening Conference (All SDR teachers - 

all day program) 

March 26, 1987 Introduction of "Newsroom" software 

(SDR-CAI/CMS teachers and selected 
regular and CAI/CMS elementary 
teachers) 

Participants completed inservice evaluation forms for both of the above 
meetings. A copy of the General Inservice Evaluation Form and a copy of the 
modified version used in the orientation meeting are found in the Appendix. 

In additioa to the types ot data specified in the evaluation design, process 
evaluation data were obtained in a series of on-site visits to regular SDR and 
CAI/CMS program classrooms by personnel from the Department of Evaluation 
Services. The purpose of these observations was to obtain teacher input 
regarding the program's functioning. Observations were conducted by Edwa r d 
Chamberlain, project evaluator, in five of the eight: (62.5%) SDR CAI/CMS units. 
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Data collected in the CAI/CMS observations included teacher responses to an 
interview instrument, CAI/C MS Evaluator's JVisi t a£ion_ _Log . Observations were 
made by Rosemary Lore, project evaluator, in two of the six (33.3%) regular SDR 
units. Data collected in the observations included teachers' responses to an 
interview instrument, Evaiuator's V isitation Log. A copy of each of the 
observation instruments is found in the Appendix, Findings from these two 
instruments are summarized in this report. The full interim reports are on file 
at the Department of Federal and State Programs (Chamberlain, 1987b; Lore, 
1987). 

Major Findings 



Due to the fact that the 1986-87 SDR program contained two treatment groups 
(regular instruction group and CAI/CMS group), data on enrollment/attendance and 
achievement testing are reported below in two ways. These data are first 
presented for the overall program regardless of treatment group. Ihe second 
presentation compares the two treatment groups in regard to 
enrollment/attendance data and achievement test data. 

In interpreting the pretest-posttest achievement data, the reader should be 
aware of the pupil selection process. Previous norm-referenced reading 
achievement data and staff recommendations were used to select and enroll pupils 
tor tne SDR program. To be eligible for the program (Objective 1.1) the pupil 
had to score at or below the 36th percentile on the selection test. Once the 
elijibility list was established, pupils were selected in order of their test 
scores with the lowest scoring pupils selected first. following enrollment, 
pupils were pretested on the CTBS Reading Comprehension subtest, Level J Form U. 

Pupil Census Information 

During the 1986-87 school year the SDR program serveu 809 pupils. Of the 
809 pupils, 736 (91.0%; were ninth-graders, and 73 (9.0%) were tenth graders. 
Of the 809 pupils, 430 (53.2%) attended the minimum number of days (103) to meet 
the 80% attendance criterion level contained in Objective 1.1. This was a 
decrease of 8.3% under last year's figure of 61.5%. A breakdown by grade level 
showed that 297 (53.9%; of the ninth-graders, and 33 (45.2%) of the 
tenth-graders met the attendance criterion. The average number of days of 
enrollment and attendance for program pupils was 105.3 and 86.8 respectively. 
The overall attendance rate for the program (total days of attendance divided by 
total days of enrollment) was 82.4%, as compared to 84.7% last year. The 
average daily membership was 660.5 , which was an average of 47.2 pupils per 
teacher as compared to 53.3 pupils per teacher in last year's program. Table 1 
contains the pupil attendance data. 

Of the 809 pupils served by *the program, teachers rated 253 (31 .3%) as 
making much progress, 321 (39.7%) as r.aking some progress, 144 (17.8%) as making 
little progress, and 91 (11.2%) as making no progress. This was measured by an 
item on the Pupil Census Form which required a subjective response from the 
project teachers, regarding their pupils' progress as they exited the SDR 
program. 

The evaluation sample of 357 pupils consisted of those pupils who met three 
criteria: attended 80% (103; of the 129 program days, received both a pretest 
and a posttest with the CTBS, and were judged to be English speaking. Of the 
357 pupils in the evaluation sample, 331 pupils were in grade 9 and 26 pupils 
were ir grade 10. 
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Table 1 

Number of Pupils Served; Averages for Days of Enrollment, Days of Attendance, 
Daily Membership and Hours of Instruction Per Week; and 
Pupils Attending 80% of Days 
Reported by Grade Level 
986-87 



Grade 



9 

10 



Pupils 
Served 



736 
73 



Girls 



310 
40 



__Boyj3_ 
426 
33 



Days of 
En rollment 

106.6 

92.0 



Days of 
Attendance 



87.8 
75.8 



~ Daily 
Member s hip_ 

608.4 

52.0 



Hours of Instruction 
per Pupil per Week 



Pupils 
Attending 
80% of Davs 



3.5 
3.5 



397 
33 



Total 



809 



350 



459 



105.3 



86.8 



660.5 



3.5 



430 
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Stan dardized A chievement Test Informat ion 

The analyses of pretest-posttest achievement data provided minimums, 
maximums, averages or medians, and diirerences tor derived scores by grade 
level. The derived scores used in the analyses were percentiles, grade 
equivalents, and normal cutve equivalents. iNo raw score data are presented 
because pupils took a different form of the test at pretest nd posttest limes. 

Table 2 contains pretest-posttest percentix* data. The median percentile 
for the pretest was 27.0 at grade 9 and 16.0 at grade 10. The median percentile 
for the posttest was 13.0 at grade 9 and 19.5 at grade 10. These data indicate 
that neither grade approached a median percentile score of 36 at posttest time* 
Further analysis of pretest percentile distributions indicated that 113 (34.1%) 
of the ninth grade pupils in the sample scored above the 36th percentile on the 
pretest, even though they had previously qualified for the program on a 
selection test. Since the program serves mostly ninth grade , this represents 
31.7% of the overall evaluation sample of 357 pupils. 

Table 3 contain? pretest-posttest grade equivalent data. The median grade 
equivalent score decreased from 7.6 to 7.3 at grade 9 but increased from 8.5 to 
8.9 at grade 10. 

The presentation of achievement data thus far has included results from the 
analyses of percentiles and grade equivalents. Both percentiles and grade 
equivalent scores provide comparative information but are not equal units of 
measure. Caution Is advised in drawing conclusions about program impact from 
any of the scores above. Normal curve equivalents (NCE's) are generally 
considered to provide the truest indication of pupil growth in achievement, 
since they provide comparative information in equal units of measurement. Data 
for NCE's are presented in Table 4. 

Objective 1.1 states that the evaluation sample would be composed of pupils 
who scored below the 36th percentile on the selection test and were in 
attendance 80% of the program's treatment period. In order to meet the 
attendance criterion the pupil had to attend at least 103 days of the 6.5 month 
(129 days) treatment period. To achieve Objective l.l the average growth in 
reading achievement of pupils in the evaluation sample had to be 1.0 NCE for 
each month of the treatment period, which is an average ->f 6.5 NCE's for the 6.5 
month treatment period. 

The overall NCE change for tne program was -8.3 or an average of -1.3 NCE's 
for each of the 6.5 months of the treatment period. This negative change fell 
considerably short of the expected evaluation criterion of 1.0 NCE gained for 
every month the pupils were in the program. A negative change of -8.9 NCE's, or 
-1.4 NCE's per month, occurred in grade 9. In grade 10 there was a positive 
change of 0.2 NCE points, or 0.0 NCE's per month. The sample size at grade 10 
was 26 pupils. 

It should be noted that NCE scores are based on percentiles, which compare 
the pupil's performance in relation to the general population. No change in NCE 
score would indicate that pupils have progressed at their normal rate of growth 
over the school year. Even a small gain in percentile or NCE score would 
indicate that pupils have advanced over the school year at a greater rate than 
would be expected from their original position in relation to the general 
population. Table 5 contains data related to the changes in NCE scores for 
three ranges: (a) no improvement in NCE scores (0.0 or less), (b) some 
improvement in NCE scores (0.1 to 6.9), and (c) substantial improvement in NCE 
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TabLe 2 

Minimum, Maximum, Median, and Standard Deviation 
he Pretest and Posttest Percentiles 
Reported by Grade Level 
1986-87 



Pretest ^ Posttest 

Number Median Standard ' Median " sTal7<Tard~~ 

Grade Oi Pupils Min. Max. Percentile Deviation Min. Max. Percentile Deviation 



9 331 5.0 77.0 27.0 16.2 i.U 99.0 13.0 18.6 

10 26 3.0 62.0 16.0 16.7 1.0 80.0 19.5 19.9 



12 
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Table 3 



Minimum, Maximum, Median and Standard Deviation 
of the Pretest and Posttest Grade Equivalents 
Reported by Grade Levol 
1986-87 



Grade 



9 
10 



Numbe r 
of Pupils 



331 
26 



Min. 



Max. 



Pret est^ 
Med ian 
Grade 
Equivalents 



4.2 12.2 
5.1 12.2 



7.6 
8.5 



Standard 
Deviat ion 



1.7 
1.6 



Posttest 



Mi n. 



4.0 
4.2 



Max. 



12.9 
12.9 



Median 
Grade 
Equivalent 



7.3 
8.9 



Standard 
Deviat ion 



2.0 
2.1 
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Table 4 

Minimum, Maximum, Average, and Standard Deviation of the 
Pretest and Posttest Normal Curve Equivalents (NCE) 
Reported by Grade Level 
1986-8/ 



Pretest p_o_sttest_ 

Number Average Standard " ~ Average Standard Average 

Grade ot Pupils Min. Max. NCE Deviation Mia. Max. NCE Deviation Change 

9 331 15.0 66.0 37.4 11. 0 1.0 97.0 20.5 15.5 -8.9 

10 26 11. 0 56.0 32.3 11.7 1.0 68.0 32.5 15.8 0.2 

Total 357 37.0 28.8 -8.3 



16 

o 
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Table 5 



Change Categories for NCE Scores 
for Total SDR program 
1986-87 



PupiLs No improvement Some Improvement Suostantial Improvement 

in Sample (0.0 or less) (0.1 to 6.9) (7.0 or more) 



Grade 9 



Number of Pupils 
% of Pupils 



331 



266 
80.4% 



26 
7.9% 



39 

1 1.8% 



Grade 10 



Number of Pupils 
% ot Pupils 



26 



13 

50.0% 



4 

15.4% 



9 

34.6% 



Total s 



Number of Pupils 
% ot Pupils 



357 



279 
78.2% 



30 
8.4% 



48 

13.4% 



18 



19 
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scores (7.0 or more;. The data indicate that 78 (21.8%) pupils made gains in 
NCE scores. lhis means that 21. 8X ot the pupils in the evaluation sample 
progressed at a rate that was greater than normal for them. More specifically, 
4« U3.4/S) made significant improvement and 30 (8.4%) made some improvement in 
NCE scores, while 279 pupils (78.2%J of the evaluation sample made no 
improvement, as evidenced by a gain of 0.0 or decrease in NCE score. In regard 
to grade level, 13 of 26 (50.0%) tanth grade pupils showed progress, while 65 of 
331 (19.6%) of ninth grade pupils showed positive progress. 

Tables 6-10 present comparisons between the group of pupils receiving 
computer assisted instruction/ computer management system (CAI/CMS) in reading 
and the group receiving the regular program instruction. As indicated in Table 
6, there were 486 pupils served by the CAI/CMS project and 323 pupils who 
received regular reading instruction. The CAI/CMS group averaged 8.8 more days 
of attendance per pupil with an overall average of 90.3 days as compared to 81.5 
days for the regular group. The average number of days attended was greater for 
grade 9 than for grade 10 in the CAI/CMS group but greater for grade 10 than for 
grade 9 in the regular group. In the CAI/CMS group 276 of the 486 pupils served 
(56.8%) met the program attendance criterion by attending at least 103 days. In 
the regular treatment group the attendance criterion was met by 154 (47.7%) of 
the 323 pupils served. The evaluation sample of 357 pupils was comprised of 223 
pupils in the CAI/CMS group and 134 pupils in the regular group. Achievement 
data for the two subpopulations of the program are presented in Tables 7-10. 

Percentile score comparisons are presented in Table 7. In grade 9 the 
median percentile score regressed from 27.0 to 12.0 in the CAI/CMS group and 
from 31.0 to 15.0 in the regular treatment group. At grade 10 the median 
percentile progressed from 11.0 to 18.0 in the CAI/CMS group and also progressed 
from 22.0 to 25.0 in the regular treatment group. 

Table 8 presents comparisons in terms of median grade equivalent scores. 
Positive changes occurred in grade 10 of both the regular treatment group and 
the CAI/CMS group but not in grade 9 ol either CAI/CMS or the regular group. 
The median grade equivalent score decreased from 7.6 to 6.9 in grade 9 of the 
CAI/CMS group and from 8.0 to 7.6 in grade 9 of the regular group. A positive 
change occurred in grade 10, where the median grade equivalent score increased 
from 7.6 to 8.8 in the CAI/CMS group and from 8.8 to 9.1 in the regular group. 
In grade 10 the overall sample was smrller (26 pupils, 7.3%), while grade 9 
comprised the bulk of the pupils (331 pupils, 92.7%). 

As indicated earlier, NCE scores are generally considered to provide the 
most comparative information in equal units of measurement. Data for the two 
groups in terms of NCE scores are presented in Table 9. The data indicate that 
the average NCE change within the CAI/CMS group was -9.7 NCE points in grade 9, 
where there were 212 pupils in the sample, and 0.3 NCE points in grade 10, where 
there were 11 pupils in the sample. In the regular treatment group the 119 
pupils in grade 9 had an average change of -7.6 NCE points, and the sample of 15 
pupils in grade 10 had an average gain of 0.1 NCE point. Neither SDR group met 
the criterion of Objective 1.1 with a change of 6.5 NCE points, or 1.0 NCE 
points for each month of instruction. An overall comparison of the two 
treatment groups is obtained by examining the average NCE changes across grade 
levels. The average change for the CAI/CMS group was -9.2 NCE points over the 
6.5 month treatment period. The regular treatment group did somewhat better 
with an average change of -6.7 NCE points in the same 6.5 month treatment 
period. 
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Table 6 



Number of Pupils Served, Averages for Days of Enrollment, Days of Attendance, 
Daily Membership and Hours of Instruction Per Week, and 
Pupils Attending 80% of Days Reported by Grade Level 
for Pupils Receiving Reading Instruction with Computers V CAI/CMS Group) 
and Pupils Receiving Reading Instruction without Computers (Regular Group) 

1986-87 



Average 



Pupils Days of " Days of Daily Hrs. of Inst. 

Grade Served Girls Boys Enrollment Attendance Membership Per Pupil Per Week 



CAt/CMS Group 

9 450 184 266 109.4 92.1 381.7 3-5 

10 36 21 15 87.0 68.5 24.3 3.5 



Tot?l 486 205 281 107.8 90.3 406.0 3.5 



Regu lar Gro up 

9 286 126 160 102.3 81.3 226.8 3.6 

10 37 19 18 96.8 82.8 27.8 3.6 



Total 323 145 178 101.7 81.5 254.5 3.6 
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TabLe 7 



Minimum, Maximum, Median, and Standard Deviation 
of the Pretest and Posttest Percentiles Reported by Grade Level 
for Pupils Receiving Reading Instruction with Computers (CAI/CMS Group) 
and Pupils Receiving Reading Instruction without Computers (Regular Group) 

1986-87 



Grade 



Numbe r 
of Pupils 



Pretest 



C^I/ CMS Group 
9 212 
10 11 



Min. 

5.0 
3.0 



Median 
Max. Percentile 



74.0 
38. C 



27.0 
11.0 



Standard 
Deviation Min, 



Posttest 



Median Standard 
Max. Percentile Deviation 



16. C 
9.3 



1.0 
1.0 



99.0 
43.0 



12.0 
18.0 



17.6 
14.6 



Regular Group 
9 119 
10 15 



5.0 
8.0 



77.0 
62.0 



31.0 
22.0 



16.6 
17.9 



1.0 
8.0 



94.0 
80.0 



15.0 
25.0 



20.0 
22.3 
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Table 8 



Minimum, Maximum, Median, and Standard Deviation 
of the Pretest and Posttest Grade Equivalents Reported by Grade Level 
for Pupils Receiving Reading Instruction with Computers (CAT/CMS Group) 
and Pupils Receiving Reading Instruction without Computers (Regular Group) 

1986-87 



Grade 



Number 
of Pupils 



CA1/CMS Group 
9 212 
10 11 



Min. 



Max. 



4.2 
5.1 



11 .8 
9.4 



Pr e test 
Median 
Grade 
Equivalents 



Posttest 



7.6 
7.6 



Standard 
Deviat ion 



Min. 



1.7 
1.1 



4.0 
4.2 



Median 
Grade 
Max. Equivalent 



12.9 
10.0 



6.9 
8.8 



Standard 
Deviat ion 



1.9 
2.3 



Regular Group 
9 119 
10 15 



4.2 
6.5 



12.2 
12.2 



8.0 
8.8 



1.7 
1.6 



4.0 
7. j 



12.9 7.6 
12.9 9.1 



2.1 
1.6 



ERIC 
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Table 9 



Minimum, Maximum, Average, and Standard Deviation of the 
Pretest and Posttest Normal Curve Equivalents (NCE) Reported by Grade Level 

for Pupils Receiving Reading Instruction with Computers (CAT/CMS Group) 
and Pupils Receiving Reading Instruction without Computers (Regular Group) 

1986-87 



Number 
Grade of Pupils 



C^l/j:MS_Group 

9 212 
10 11 



15.0 
11.0 



Pretest 



64.0 
44.0 



37.1 
25.6 



11.1 
8.3 



Posttest 



Average Standard 
Min. Max. NCE Deviation Min. 



Average Standard Average 
Max. NCE Deviation Change 



1.0 
1.0 



97.0 
46.0 



27.4 
25.9 



15.0 
16.4 



-9.7 
0.3 



Total 



223 



Regular Group 

9 119 
10 15 



36.5 



27.3 



-9.2 



15.0 66.0 38.0 
20.0 56.0 37.1 



11.0 
11.6 



1.0 
21.0 



83.0 
68.0 



30.4 
37.3 



16.1 
13.9 



-7.6 
0.1 



Total 134 37.9 31.2 -6.7 
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Table 10 compares the CAI/CMS and regular groups in regard to numbers and 
percents of pupils who evidenced no improvement, some improvement, and 
substantial improvement, as previously defined. The data indicate that 37 
pupils {l/.b/o) or the regular group pupils made positive gains in NCE scores, 
while 41 pupils (18.4)0 oi CAI/CMS groups did so. Positive gains in the regular 
aroup included 24 pupils (17.9%) who made substantial improvement and 13 pupils 
(9.7%) who made some improvement. Positive gains in the CAI/CMS group included 
24 pupils (10.8%) making substantial improvement, and 17 pupils (7.6%) making 
some improvement. 

Inservice Evaluation Inform a tion 

Objective 2.1 stated that program personnel would be provided at least two 
inservice sessions and that at least 80% of the personnel attending each session 
would rate the session as valuable in providing information that would assist 
them in carrying out their program responsibilities. A total of two inservice 
meetings was provided by the Department of Federal and State Programs. All SDR 
teachers were given the opportunity to attend the orientation meeting of 
September 2, 1986, but the meeting of March 26, J 987 , was limited to CAI/CMS 
teachers. A modified version of the General Inservice Evaluation Form was used 
for the orientation meeting while the other inservice meeting was assessed using 
the regular Ge, ^ai Inservice Evaluation Form. 

Analysis of teachers' ratings to individual inservice meetings indicated 
that 71.4% of the teachers attending the overall program meeting of September 2, 
1986, either agreed or strongly agreed that the meeting was valuable in 
assisting them in their program. The one other inservice meeting was held for 
SDR-CAI/CMS teachers and received favorable ratings by 100% of the participants. 

Objective 2.1 states that program personnel will be provided at least two 
inservice sessions such that at least 80% of the inservice participants will 
rate each session as valuable in providing information that will assist them in 
carrying out their program responsibilities. Although there were two inservice 
meetings provided, as per the criterion of the objective, it should be noted, 
again, that only one meeting (orientation) was provided to all program 
personnel; the other was provided for SDR-CAI/CMS teachers. The 80% criterion 
was not attained since one of the two inservice meetings received favorable 
ratings dv less than 80% of the participants, at 71.4%. 

Table 11 contains a summary of the combined teacher ratings for both of the 
inservice programs. In this combined rating, 88.9% of the participants agreed 
or strongly agreed that the information in the meetings would assist them in 
their program. Ratings were based on the following five-point scale: 

1 = Strongly Disagree 

2 = Disagree 

3 « Undecided 

4 = Agree 

5 = Strongly Agree 
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Table 10 

Change Categories for NCE Scores for Total SDR Program Reported by 
Grade Level for Pupils Receiving Reading instruction with Computers 
(CAI/CMS Group) and Pupils Receiving Reeling Instruction 
without Computers (Regular Group) 
1986-87 



Pupils No Improvement 

in Sample (0.0 or less) 



Some Improvement Substantial Improvement 
(0.1 to 6.9) (7.0 or more) 



CAI/CMS Group 

Grade 9 

Numbei. of Pupils 
% ot Pupils 



212 



177 
83.5% 



15 
7.1% 



20 
9.4% 



Grade 10 

Number of Pupils 
% of Pupils 



11 



5 

45.5% 



2 

18.2% 



4 

36 .4% 



Total 

Number of Pupils 
% of Pupils 



223 



182 
81.6% 



17 
7.6% 



24 

10.8% 



Regu lar Group 



Grade 9 

Number of Pupils 
% of Pupils 



119 



89 

74.8% 



11 
9.2% 



19 

16.0% 



Grade 10 

Number of Pupils 
% of Pupils 



15 



8 

53.3% 



2 

13.3% 



5 

33.3% 



Total 

Number of Pupils 
% of Pupils 



134 



97 

72.4% 



13 
9.7% 



24 

17.9% 
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Table 11 

Average Response and Percent of Response 
For Reactions to Inservice Statements 



Percent 



Number Average SA A U D SD 
Scatements R esponding Response (5) (4) (3) (2) ( 1 ) 

1. I think this was 
a very worthwhile 

meeting. 18 4.3 61.1 22.2 5.6 11.1 0.0 

2. The information 
presented in the 
meeting will assist 

me in my program. 18 4.3 55.6 33.3 0.0 11.1 0.0 

3. There was time to ask 
questions pertaining 

to the presentation. 17 4.5 64.7 29.4 0.0 0.0 5.9 

4. Questions were 

answered adequately. 16 4.4 68.8 18.8 6.3 G.O 6.3 



Open-ended comments on the General Inservice Evaluation Form asked 
participants to comment about the most and least valuable parts of the meetings 
and about information they would like to have covered in future meetings. The 
evaluation reports on individual sessions have been forwarded to the Department 
of State and Federal Programs and are available on request. 

In regard to the most valuable parts of the inservice meetings, teachers 
liked the exhibits and materials session of the orientation meeting. They also 
liked meeting with experienced program teachers and meeting new program 
teachers. The March 26 inservice which featured hands-on experience with the 
"Newsroom" computer software was popular. For the question dealing with the 
least valuable part, teachers felt that the orientation meeting didn't meet the 
needs of secondary labs (they said sessions were geared to elementary and middle 
schools). In addition, they felt that there was not enough time for viewing 
exhibits and materials. Four of the suggestions for future meetings dealt ;- 7 ith 
the desire to have meetings that deaJ specifically with secondary programs, 
with one teacher saying: "If sessions are not to be addressed to high school 
level, let us use the time for ordering materials." Four other suggestions for 
inservice were for more programs to help students develop writing skills, two 
were for inservice about computer lab materials suitable for high school level, 
and two were for new computer software inservice. 

It is concluded that the second criterion of Objective 2.1 was not attained, 
because one of the two meetings was rated as valuable in carrying out component 
responsibilities by less than the requisite 80% of the participants. The other 
remaining meeting , however, was favorably rated by 100% of the participants. 
When a combined rating of both inservice meetings wac computed, an overall 
average of 88.9% of the participants rated the inservice sessions as valuable in 
carrying out program responsibilities. 
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S chool V isitation Inf or ma t ion 

In addition to the types of data specified in the evaluation dosign, process 
evaluation data were obtained by means of on-site visits. Five CAI/CMS teachers 
and two of the regular SDR teachers were interviewed during the school year 
during visits by project evaluators. These visits occurred in March 1987. 

A locally developed instrument, the CAI/CMS E valuator' s Visita t ion Log , was 
used in interviewing teachers in the CAI/CMS portion of the program. The 
interviews dealt with general program concerns, as well as items specific to the 
CAI/CMS setting. The interview sample consisted of five of the eight CAI/CMS 
teachers in the program. 

Although some technical difficulties had occurred with computers, nearly all 
had been satisfactorily resolved. Teachers reported that there was no problem 
obtaining printed materials to go with their program. The materials are 
promptly supplied by Prescription Learning Company, Computer effectiveness for 
diagnosis and instruction was rated on a five-point scale, with average ratings 
of 3.8 for diagnosis and 4.2 for instruction. 

Teachers gave testing procedures an average rating of only 2.4 on a 
five-point eating scale. The chief problem with testing, as perceived by 
CAI/CMS teachers, was the difference between testing conditions in pretest and 
posttest administrations. Although the pretest is given by the prcgrara teacher 
in the lab, the posttest for grade 9 is administered in a larger group as part 
ot Districtwide Testing. There was also a feeling that standardized tests do 
not tell the whole story of pupil progress. 

Program goals and objectives perceived by CAI/CMS teachers included the 40 
skills objectives abound which the Prescription Learning program is centered, 
raising the pupil's reading level, and getting pupils to enjoy reading. 
Establishment of classroom policies and rules, use of reward systems, and 
organized work schedules were among the techniques used by teachers to maximize 
the use of learning time. Teachers reported that they convey their expectations 
for learning and behavior verbally to the pupils, and that praise and 
encouragement are used to provide recognition and feedback to r.he pupils. 
Various methods and materials were cited as helpful in teaching reading 
comprehension. Materials mentioned by teachers included Prescription Learning 
materials, newspapers, and the Barnell-Loft Specific Skills Series. The most 
frequently mentioned tools for diagnosis were those provided by Prescription 
Learning Company. 

In the regular SDR visitations, the locally developed instrument, 
^valuator's Visitation Log , was centered around questions designed to gather 
information about the major facets of the SDR program. There were open-ended 
questions covering such aspects of the program as goals and obj ectives and 
coordinating with the regular classroom teacher. Twelve rating scales were used 
to gather information suitable to the use of descriptors. Some highlights of 
the responses to the open-ended questions in the interviews follow: Teachers' 
major goals and objectives are to increase pupils' comprehension levels as much 
as possible; use informal and formal assessment to diagnose pupils' strengths 
and weaknesses so that they can prescribe a program for them; use varied 
materials and methods for teaching comprehension so that pupils don't get bored; 
make maximum use of learning time through discipline, goal setting, clear 
directions, and daily routines. Teachers let pupils know they had high 
expectations for them by "telling them," and giving them challenging work; 
providing feedback and recognition bj giving praise and tangible rewards; 
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monitoring student progress by conferring with classroom teachers and grading 
papers. Teachers expressed concern about not having more inservice - regular 
SDR teachers had only one meeting, which they felt did not address their 
concerns. They spoke of the difficulty of coordinating their programs with the 
classroom teachers because of lack of common conference times and location of 
rooms. Another area of concern was the poor response of parents to teachers' 
actempts at irwolveraent # 

Additional questions were rating scales and dealt with the importance of 
coordinating instruction with the classroom teacher, communication with 
classroom teachers, parent responses to program teachers' efforts at program 
involvement, environmental temperature and noise level, selection procedures, 
scheduling, testing procedures , evaluation feedback, facilities , space, and 
materials. Teachers were asked to rate these facets of their programs on a 
five-point rating scale where the lowest descriptors (Unimportant, Very Poor, 
inadfc^udte) were rated as "l" a;id the highest descriptors (Very important, Very 
Good, Very Adequate) were rated as "5." Facets receiving a rating of "3" or 
higher were: selection procedures (3.5), scheduling (4.0), evaluation feedback 
(4.5), communication with classroom teachers (3.0), environmental temperature 
(3.0), and facilities (3.0). There was great concern about parent response to 
program teachers' efforts at program involvement (1.5) and concern about the 

environmental noise level (l.5). A copy of the Evaluator's V isitat ion Log 

appears as part of the Appendix. ™~ ~~ " 

Cost-Benefit Analysis Informatio n 

The program evaluation included one further analysis not in the original 
evaluation design: a cost-benefit analysis (Chamberlain, 1987a) comparing the 
CAI/CMS group and the regular group. This analysis is summarized in Table 12. 
Costs included in the analysis included teacher salaries and the contract cost 
for Prescription Learning Laboratory Reading Labs. Normal supplies and 
incidental costs were not known in regard to the two groups but were assumed to 
be evenly distributed. Any error of cost estimate resulting from unknown costs 
would probably be in the direction of underestimating the cost for the Regular 
group, since most instructional materials for the CAI/CMS group were included in 
the Prescription Learning Laboratory contract costs. The cost-benefit analysis 
indicated that the cost per pupil was $286.39 more per pupil in the CAI/CMS 
group than in the regular treatment group when computed on average daily 
membership. However, the u3e of computers enabled CAI/CMS teachers to serve an 
average of 8.4 more pupils per teacher than in the Regular group (based on 
average daily membership). 

In the CAI/CMS group 56.8% of the pupils served attained the program's 
attendance criterion, compared to 47.7% of pupils in the regular treatment group 
who met the attendance criterion. The evaluation sample, which depends heavily 
on attainment of the attendance criterion, was comprised of 45.9% of all pupils 
served in the CAI/CMS group compared to 41 .5% of all pupils served in the 
regular treatment group. As noted earlier, there was a negative change in NCE 
scores in both groups. The average change for the regular group was -6.7, while 
the NCE change for the CAI/CMS group was -9.2. 



Summary 

The Secondary Developmental Reading Program is an individualized learning 
program designed to assist secondary pupils who are having reading problems. 
During the 1986-87 school Year, 14 project teachers working in 13 senior high 
schools served a total of 809 pupils in grades 9-10. 
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Table 12 



Cost-Benefit Analysis for 1986-87 Secondary Developmental Reading Program 
Comparing Group Receiving Computer Assisted Instruction/Computer 
Management System tCAl/CMS) and Group Receiving Regular Program Instruction 



Program 



Number 
of 

Teachers 



Program Cost 



Total 



Per 
Teacher 



Average 
Daily Membership 

In Per 
Program Teacher 



Cost 

Per 

Pupil 



Percent 

of 
Pupils 
Meeting 
Attendance 
Criterion 



Ratio of 
Sample 

to 
Pupils 
Served 



Average 
NCE 
Gain 



SDR-PLL 
(grades 9-10 
with CAI/CMS) 



479,390.08 59,923.76 



406.0 



50.8 



1180.76 



56.8% 



45.9% 



-9.2 



SDR 

(grades 9-10 
Regular group) 



227,617.56 37,936.26 254.5 



42.4 



894.37 



4> .7% 



41.5% 



-6.7 
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The program had two objectives. Objective 1.1 stated that pupils who 
attended ttU% ot the 6.5 month treatment period would show an average gain in 
reading of 1.0 NCE for each month, which is an average gain of 6.5 NCE's overall 
(6.5 months x 1 .0 NCE). This objective was not attained. The program showed an 
overall negative change of -8.3 NCE points for the 6.5 month treatment period, 
or -1.3 NCE's per month. In grade 10, the NCE gain was 0.2 NCE for the 
treatment period, or 0.0 NCE per month. The negative change in grade 9 was -8.9 
NCE's for the treatment period, or -1.4 NCE's per month. 

Teacher perceptions of pupil progress, as measured by an item on the Pupil 
Census Form, suggested that they felt there was more pupil progress than test 
scores indicated. Of the 809 pupils served by the program, teachers rated 253 
(31.3%) as making much progress, 321 (39.7%) as leaking some progress, 144 
as making little progress, and 91 (11.2%) as making no progress. 

Objective 2.1 stated that program personnel would be provided at least two 
inservice meetings and that at least 80% of the personnel attending each meeting 
would rate the meeting as valuable in providing information that would assist 
them in carrying out their program responsibilities. There was a total of two 
inservice meetings provided by the Department of Federal and State Programs. 
Regular and CAI/CMS program teachers were given the opportunity to attend the 
orientation meeting. The other meeting was for CAI/CMS teachers. One of the 
meetings was ^ated as valuable in carrying out program responsibilities by more 
tuan Lhe requisite 8LU of participants. However, objective 2.1 was not attained 
because one of the two sessions was rated as valuable in carrying out component 
responsibilities by less than the requisite 80% of the participants. 

Program teacher interviews during school visitation process evaluation 
indicated that teachers were concerned about the paucity of inservice meetings. 
Regular SDR teachers had only one meeting, whicli they felt did not address their 
concerns. Also, teachers spoke of the difficulty of coordinating their programs 
with the classroom teachers because of lack of common conference times and 
location of rooms. Another area of concern was the poor response of parents to 
teachers' attempts at involvement. There was a perception among CAI/CMS 
teachers that the ninth grade posttest was given under less favorable conditions 
than was the pretest. 

The CAT/CMS project was located in eight high schools. The computer 
assisted units served 486 pupils, while 323 pupils were served in the Regular 
group. Neither the CAI/CMS project group nor the group receiving regular 
program instruction attained the achievement criterion. The CAI/CMS group had a 
negative change of -9.2 NCE's in a 6.5 month period, while the Regular group had 
a negative change of -6.7 NCE's. 

During process evaluation teachers expressed a belief that one cause for the 
poor NCE growth of SDR pupils was differences in test administration 
(pretest-small group; posttest-large group). It should be noted that in 
1984-85, the first year the CTBS was adopted as the pretest and posttest 
instrument, SDR pupils did not meet criterion - even when program teachers 
controlled the testing environment. In that year SDR pupils made an overall 
average negative change of -2.5 NCE points. 

A cost-benefit study indicated that cost per pupil was greater and NCE gains 
smaller in the CAI/CMS group than in the Regular group. Based on average daily 
membership, the cost per pupil was $286.39 more in the CAI/CMS group than in the 
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Regular group. The Regular group made a negative average NCE change of -6.7, 
while the CAI/CMS group made a negative average change of -9,2. However, 
CAI/CMS teachers were able to serve an average of 8.4 more pupils per teacher 
than in ';he Regular group, uased on average daily membership* Attendance also 
was soaewhat better in the CAI/CMS gtoup than in the Regular group as judged by 
the percent of pupils attaining the program's attendance criterion of attending 
80% of the days ia a 6.5 month treatment period. The percent of pupils 
attaining this attendance criterion was 56.8% in the CAI/CMS group, as compared 
to 47.7% in the Regular group. 

During the 1986-87 school year, the Secondary Developmental Reading Program 
experienced problems in several areas. 

1. Pupil achievement : In terms of NCE scores, 78.2% of the pupils 
in the sample showed no improvement ; 8.4% showed some improvement 
but did not attain the achievement criterion of 1 .0 NCE per 
month; and 13.4% met the achievement criterion. 

2. Pupil attendance : The average pupil was enrolled in the program 
105.3 days but attended only 86.8 days. The ovetall attendance 
rate (total days of attendance divided by total days of 
enrollment) was 82 .4% , which averages out to an absence rate of 
• 88 day per week. Part of the problem appears to be that the 
average pupil was not enrolled in the program long enough to 
expect meeting the requisite number of days of attendance (103 
days) to attain the attendance criterion. 

3. Inservice: The program had two inservice meetings this year, as 
per Objective 2.1. However, only one meeting was available to 
all program teachers, and this meeting did not attain a positive 
rating bv the requisite 80% of program teachers. 

C oordination of program with classroom teacher : Process 
evaluation indicated that program teachers had difficulty 
coordinating their programs with the classroom teachers because 
of lack of conference times and location of rooms. 

5. Parent in volvemen t : Process evaluation indicated low parent 
response to program teachers' efforts at encouraging parent 
participation in the program. 

R ecommendations 

Since the Secondary Developmental Reading Program is to be continued for 
he 1987-88 school year, consideration should be given to the following: 

1. The program should become an elective course 'or those pupils who 
scored at or below the 36th percentile on a selection test • All 
eligible pupils should be approached and made aware of the 
program opportunity. Pupils would receive one-half credit for 
the year contingent on their fulfillment of a signed contract to 
attend 80% of the program c\ys, and upon the program teacher's 
judgment of pupil effort. 
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2. More inservice should De provided for program personnel, and 
should be geared specifically to the high school level. Methods 
of teaching reading comprehension should be especially stressed 
in the inservice. 

3. Coordination of program instruction with classroom instruction 
should be facilitated by time for communication between program 
and classroom teachers at regularly scheduled meeting times. 
Program instruction should be directed toward success in the 
content area where the pupil needs the most help. 

4. New ways for encouraging parent involvement need to be studied. 
Efforts such as having evening meetings, visiting homes, or 
rewards such as door prizes might be some possible approaches. 
An inservice where teachers could "brainstorm 11 might be fruitful 
in generating creative solutions to this problem. 

5. Further expansion of the CAI/CMS project is not warranted at this 
time until greater effectiveness can be demonstrated. If the 
CAI/CMS program is to continue at the high school level , a 
careful review of the present program is indicated. Problem 
areas should be identified, and solutions proposed. Positive 
aspect* oi the program should also be identified. One approach 
that is suggested would be to form a focus group comprised of all 
eight CAI/CMS teachers. 

6. Review selection procedures, correlation of course content to 
system's Course o f Study , instructional methods, class size, and 
test content to determine why pupils are not showing desired 
growth. 

7. School administrators and staff should take the responsibility of 
assuring an optimum testing environment by not scheduling 
unsuitable activities during testing weeks and by adjusting class 
schedules to accommodate the length of the tests. 

8. Conditions for the pretest and for the posttast should be as 
comparable as possible with all examiners trained to give the 
tests per instructions in the Examiners' Manuals. Pupils should 
not be tested in groups larger than recommended by the testing 
company, 

9. A study should be made to assess the comparability between the 
standard and customized versions of the ninth grade test. The 
study could be conducted as part of Districtwide Testing, using a 
representative sample of the district's ninth grade population. 

10. Extensive review of this program should be continued in regard to 
policies and procedures , selection, scheduling, attendance 
patterns, test administration, and achievement test scores. The 
review should determine whether the model for the program should 
be continued in its present form, modified, or discontinued. 
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COLUMBUS PUBLIC SCHOOLS - Columbus. Ohio PUPIL CENSUS FORM 



LAST NAME 



FIRST NAM! 



TEACHER NUMBER 



J 



SCHOOL H R GRADE 

USE A NUMBER 2 PENCIL. ERASE COMPLETELY WHEN MA V 'NG CORRECTIONS. 

WAS THIS A "NON-ENGLISH SPEAKING" STUDENT? 
YES NTj 
* * 

ooooooooooooooooooooooooooooooooooooo 

DID THIS PUPIL BECOMTQUALIFIED FOR A SPECIAL ED. PROGRAM? 1 



TES~ 
* 



ooooooooooooooooooooooooooooooo o ooooo 

HOW DID YOU FEEL THIS PUPIL PROGRESSED WHILE IN YOUR PROGRAM? | 

MUCH PROGRESS SOME PROGRESS LITTLE PROGRESS NO gROGRESS 

oooooooooooooooooooooooooooooo o oooooo 

— - I 



ooooooooooooooooooooooooooooooooooo o o 



oooooooooooooooooooooooooooooooo o o o o o 



oooooooooooooooooooooooooooooooooooo o 
I 



ooooooooooooooooooooooooooooooooooooo 

I 



ooooooooooooooooooooooooooooooooooooo 

— • I 



ooooooooooooooooooooooooooooooooooooo 

r ] 
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1986-87 
Teacher Census Form 



Social Security Number 



Name 



School Assignment 



Program Code 
Cost Center 



Circle only the program you are in: 

ECIA Chapter 1 Programs: 
( 1 ) ADK 

12) CLEAR-Elementary (1-5) 

(3) CLEAR-Elementary-CAI (4-5) 

(4) CLEAR-Middle School (6-6) 

(5) CLEAR-Midd le-CAI (6-8) 



DPPF Programs: 

(6) SDR (9-10) 

(7) SDR-CAI (9-10) 

(8) HSCA 

Other (Specify) 



a Number of Years of Teaching Experience 

^Number of Years of Title l/Chapter 1 Teaching Experience 

C I am certified in reading as indicated by the subject area on my teaching 
certificate. 



Yes 



No 



Highest College Degree Received 



Full-Time Employee^ 
or 

Pact -Time Employee 



a Total all years of experience, including those which r ,y have occurred 
outside of the City of Columbus. Please include present s chool year. 

b l. For every full year taught in Title l/Chapter 1 give yourself 10 
months experience. Please include the present schoc 1 year. 

2. B">r every summer term you taught in Title I give yourself two 
months experience. 

3. Add in any miscellaneous experience, a part-year perhaps. 

4. Add the totals for 1, 2, and 3 and divide by 10. Place the 
resulting quotient in the blank for question b above. 

Certification is defined as having one of the following: 

1. reading specified on Bachelor degree. 

2. reading specialist certificate. 

3. M.A. in reading as a subject. 
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GENERAL INSERVICE EVALUATION FORM 



Inservice Topic: 
Presenter(s) : 



Date: (e.g., 03/05/86) 

Session: a.m. and/or p.m. 

Circle only the program you are in: 

ECIA Chapter 1 Programs: 

(i; ADK 

(2) CLEAR- Reading Recovery 

(3) CLEAR-ElemenLary (1-5) 

(4) CLEAR-Elementary-CAI 

(5) CLEAR-Middle School (6-8) 

(6) CLEAR-Middle School-CAI 

DPPF Programs: 
(7; SDR (9-10) 

(8) SDR-CAI 

(9) HSCA 

Other ^Specify) 



Circle the number that indicates the extent to which you agree with statements 1-4. 



Strongly Strongly 
Agree Agree Undecided Disagree Disagree 



1. I thiik this was a very worthwhile 
mee t ing • 



2. The information presented in ''lis 
meeting will assist me in my 

program. 5 4 3 2 1 

3. There was time to ask questions 

pertaining to the presentation. 5 4 3 2 1 

4. Questions were answered 

adequately. 5 4 3 2 1 

5. What was the most valuable part of this meeting? 



6. What was the l ease valuable part of this meeting?^ 



7. What additional information or topics would you like to see covered in future 
meetings? 
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ECIA CHAPTER 1 
ORIENTATION INSERVICE EVALUATION FORM 
September 2, 1986 



Cii.de only the program you are in: 

ECIA Chapter 1 Programs: 

( 1 ) Ai)K 

(2) CLEAR-Elementary U~5) 

(3) CLEAR- Element ary-CAI (4-5) 

(4) CLEAR-Middle School (6-8) 
(5; CLEAR-Hiddle-CAI (6-8) 

Circle the number that indicates the extent to which you agree with statements 1-4, in 
rating the overall day of inservice. 



DPPF Programs: 

(6) SDR (9-10) 

(7) SDR-CAI (9-10) 

(8) HSCA 
Other (Specify) 



Strongly 



Strongly 



Agre e Agree Undeci ded Disagree Disagree 



1. I think this was a very worthwhile 
inservice. 

2. The information presented in this 
inservice will assist me in my 
program* 

3. There was time to as*c questions 
pertaining to the presentations. 

4. Questions ^re answered adequately. 



5 
5 



4 
4 



3 
3 



2 
2 



Circle the number that indicates how you would rate each of the following portions of 
today's inservice in regard to interest and usefulness of presentations. 



Superior Excellent 



5. Large Group Session 

a. Interest 

b. Usefulness 

6. Commercial Exhibits 

a. Interest 

b. Usefulness 

7. Min^session with main speaker 

a. Interest 

b. Usefulness 



5 
5 

5 
5 

5 
5 



4 
4 

4 
4 

4 
4 



Good 

3 
3 

3 
3 

3 
3 



Fair 

2 
2 

2 
2 

2 
2 



Poor 



ERICrALSRVCS/CHAPTER 1/0RIEN86 



45 



29 



Superi or Excellent Good Fair poor 

tf. Chapter 1 mini-session 

a. Interest 5 4 3 2 

b. Usefulness 5 4 3 2 

c. Clarity of instructions 5 4 3 2 

9. Evaluation Presentation 

a. Interest 5 4 3 2 

b. Usefulness 5 4 3 2 

c. Clarity of instructions 5 4 3 2 
10. What was the most valuable part of this meeting? 



11. What was the least valuable part of this meeting? 



12. What additional information or topics would you like to see covered in future 
meetings? 
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Columbus Public Schools 
ECIA Chapter 1 and DPPF-SDR Programs 
EVALUATOITS VISITATION LOG 



CLEAR-Elera (1-5) 

CLEAR- Mid (6-8) 

SDR (9-10) 

School Date 

Program Teacher Evaluator 

A. Questions 1-11 (Open-ended Comments) 

1. Does your program have goals and objectives? Explain. 



2. What diagnostic test(s) or methods do you use? 



3. How have the results of the diagnostic test(s) or methods been helpful 
in adjusting your approach to instruction? 



4. What instructional methods and materials have you found particularly 
effective in Improving reading comprehension? 
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5. 



What do you do to promote the maximum use of academic learning time? 
(Time on Task) 



6» How do you let pupils know that you have high expectations for their 
learning and behavior? 



7. How do you give recognition and provide feedback to pupils? What seems 
to work best? 



8» How do you monitor student progress? 



9. Did your pupils make as much progress in reading last year as you had 
hoped? If yes, why; if no, why not? 



10. In what way has current research and/or inservice been helpful in your 
approach to instruction? 



11* What, if anything, do you regularly do to coordinate your program with 
the reading program the pupils receive from their classroom teacher? 
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Circle the number that indicates the extent to which the program teacher agrees. 
B. Questions 12-23 (Rating Scale) 

Very Important Unimportant 

4 3 2 1 



12. Importance of Coordinating 

Instruction with Classroom Teacher 5 



13. Communication with Classroom 
Teachers 



Very Good 
5 



Very Poor 



14. Parent Response to Your Efforts 
at Fcogram Involvement 



Very Good 
5 



Very Poor 
2 1 



15. Environmental Temperature 



1 



16. Environmental Noise Level 



17. Selection Procedures 



Very Adequate 
5 4 



Inadequate 
2 1 



18. Scheduling 5 4 3 2 1 

19. Testing Procedures 5 4 3 2 1 

20. Evaluation Feedback 5 4 3 2 1 

21. Facilities 5 4 3 2 1 

22. Space 5 4 3 2 1 

23. Materials 5 4 3 2 1 
C. Question 24 (Yes No) 

24. Do you have any Pupil Census Forms or Add Forms you would like to give me today? 
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Columbus Public Schools 
ECIA Chapter 1 and DPPF-SDR Programs 
CAI/CMS EVALUATOR'S VISITATION LOG 



l.Type of school (check one ) 



^Elementary 
"Middle School 
"High School 



Type of computer 



Apple 
"PET 

[Dolphin 
"Other 



Company Servicing Computers 
Prescription Learning 



Houghton-Mifflin 

Other 

None 



2. Computer Technical Difficulties 



a. Minor difficulties 

b. Major dirficulci.es 



Were the Problems 
Resolved Satisfactor ily 



Freque ncy of Occurrence 
Seldom 

or Never Occ asionally Frequently Yes No Partially 



3. Are there any problems getting printed materials that go with the program? 



4. Does your program have goals and objectives? Explain. 
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5. How important is it to coordinate instruction with the classroom teacher? 

» 3 2 



Very Important 
5 



Unimportant 
1 



6. What, if anything, do you regularly do to coordinate your reading program with the 
reading program the pupils receive from their classroom teacher? 



7. In general, how would you rate the degree of communication between you and the 
classroom teacher? 



Very Good 
5 



Very Poor 
1 



8. In general how would you rate parent response to your efforts at parent involvement? 



Very Good 
5 



Very Poor 
1 



9. How would you rate the following? 



Selection Process 

Scheduling 

Testing Procedures 

Evaluation Feedback 

Facilities 

Space 

Materials 

Computer Effectiveness 

1. For Diagnosis 

2. For Instruction 

Environmental Temperature 
Environmental Noise Level 



Very Adequate 
5 
5 
5 
5 
5 
5 
5 

5 
5 



Very Good 
5 
5 



4 
4 
4 
4 
4 
4 
4 

4 
4 

4 
4 



3 
3 
3 
3 
3 
3 
3 

3 
3 

3 
3 



2 
2 
2 
2 
2 
2 
2 

2 
2 

2 
2 



Inadequte 



Very 



Poor 
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10. What do you do to promote the maximum use of academic learning time (Time on Task)? 



11. How do you let pupils know that you have high expectations for their learning and 
behavior? 



12. How do you give recognition and provide feedback to pupils? 



13. What instructional methods and materials have you found particularly effective 
in improving reading comprehension? * 



14. Did your pupils make as much progress in reading last year as you had hoped? If 
yes, why; if no, why not? 
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15» What diagnostic test(s) or methods do you use? 



It. How have diagnostic testes) or methods been heip f n; adjusting your approach to 
instruction? 



17*. How do you ir^nitor student progress? 



18. In what way has current research and/or inservice been heipfui in your approach to 
ins truce ion? 



1^ Do you have any P-pil Census Forms or Add Forms you wouid like to give me today? 
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CHAPTER i AND DPPF TESTING OBSERVATION SCALE 



Observer School Date 

Time of Day Day of Week Number of Students 

Program Grade Test 



Testing Environm e n t 

Use the following key to rate the conditions of the testing envirc nment. 

VG ■ Very Good P = Poor 

G - Good VP = Very Poor 

A - Acceptable 



Lighting in the testing area 


VG 


G 


A 


P 


VP 


Space for each studeit 


VG 


G 


A 


P 


VP 


Sound or noise level 


VG 


G 


A 


P 


VP 


Temperature 


VG 


G 


A 


P 


VP 



Type of Room: Classroom Library Lunchroom 

Other _ 

Test Directi ons 

How were the directions given? Read by Proctor Written on the Board 

Other 



1. Audibleness of the instructions VG G A P VP 

2, Extent to which proctor provided for 

students' questions VG G A P VP 

^. The clarity of proctor(s) answers to 

students' questions VG G A P VP 

4. Clarity of directions for marking answer VG G A P VP 

5. Extent to which proctor followed direc- 
tions In the examiner's manual VG G A P VP 

6. Attitude of the proctor toward the 

testing process VG G A P VP 

7. Accuracy ot the procedure for timing 

the test VG G A P VP 
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Test ing Materials 

During the testing session the following materials were available: 

Lb A test booklet for each pupil with answer 

sheet where applicable Yes No NA 

2b A copy of the test booklet for demonstration 

purposes Yes No NA_ 

3. Teacher's Directions Yes No NA 

4. A pencil with eraser for each pupil, plus 

extras to cover breakage Yes No NA 

5. A stopwatch, or a watch or clock with a 

second hand, to be used for timing the tests Yes No NA 

6. A "Testing — Do Not Disturb" sign for the 

door Yes No NA_ 

7b A paper or cardboard place marker, approxi- 
mately 2" x 4" for each pupil, plus extras 
(required for kindergarten, suggested for 

grades 1-3) Yes No NA_ 

During the Tests: 

1. Proctor circulated continuously around the room 

monitoring students Yes No NA_ 

2. Proctor limited assistance to mechanical aspects 
of marking answers, clarifying directions, and 

finding right place on answer sheet Yes No NA 

3. Were there interruptions or disturbances during 
the testing period. If yes, please specify 

(what and how many times): Yes No NA 
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